1. Field of the Invention
The present invention generally relates to the processing of image sequences, and more particularly to a method of edge-directed image interpolation utilizing sub-pixel directions.
2. Description of the Background Art
Digital image information is typically represented in a series of frames which each contain an array of pixels. Information about specific image characteristics are then maintained for each pixel within the frame. Typically, these image characteristics provide intensity, and/or color information. In one image format, each pixel is represented as a mixture of three colors that are each defined by eight-bit color values. A number of standards are available for representing images in a series of frames. In certain situations, a frame may contain one or more pixels for which no image information is available. To minimize image degradation within these frames, the information for the missing pixels is generated by interpolating the characteristics of adjacent pixels.
Pixel information within a frame may be missing for a variety of reasons, including losses or boundary conditions that may result when executing a video conversion process, such as the conversion between interlaced television field signals and progressive frame signals. The interlaced scanning format is widespread and provides transmission advantages, however, it suffers from a lack of vertical resolution and often produces image artifacts such as line flicker. Therefore, interlaced images are often converted to progressive frame signals by performing an interlaced-to-progressive conversion (IPC) method. Execution of an IPC method can improve vertical resolution while reducing line flicker and other image artifacts. The conversion of an interlaced video format to a progressive format typically creates one or more rows of missing pixel data, such as near the top or bottom of a frame. Maintaining high video picture quality requires that the image characteristics of the missing pixels be replaced with an approximation of the missing pixel information that is capable of properly blending with the image so that minimal adverse viewing effects are incurred. Although pixel interpolation is utilized in the process of converting interlaced streams to progressive scan streams, it should be appreciated that pixel interpolation is generally applicable to a number of situations in which pixel information is otherwise missing, or in error. The generation of missing pixel information is typically performed by executing a pixel interpolation method within a software routine that executes on an image processing device.
One conventional pixel interpolation method, referred to as edge-directed interpolation, seeks to generate a value for a missing pixel by extending the color patterns in selected directions, expressed as color edges which exist in the surrounding pixels of an image frame. A pixel “x” for which color information is not available is shown surrounded by neighboring pixels in FIG. 1, which illustrates candidate edge directions associated with conventional edge-directed interpolation. Each of the neighboring pixels is adjacent the missing pixel with a common edge or corner. The value assigned to pixel “x” is registered by determining and then following one of the local edges defined along one of the pixel paths (a−f), (b−e), or (c−d). It should be appreciated that the illustrated pixel paths may be rotated by ninety degrees if information was missing for a segment of vertical pixels instead of the more common horizontal pixels. The selection of which local edge to use for interpolation is performed by determining which is the “least harmful” local edge direction. The term “least harmful” may be characterized by the generated pixel being created in relation to the surrounding pixels so as to have the least harmful effect on the resultant image, wherein the peak signal to noise ratio (PSNR) for the frames within a video stream is maximized, and thus less signal degradation from the original images occurs. The “least harmful” local edge is detected by finding the edge direction in which the difference between the neighboring pixel values on opposite sides of the missing pixel is at its lowest value, minimized. The interpolation is thereafter performed by finding the average of the neighboring pixels along the “least harmful” local edge. It should be appreciated that “averaging” or other normalizations described herein are performed according to the type of pixel image data being represented. For example, averaging a pixel whose color is expressed as an RGB value comprises averaging each of the separate color values of red, green, and blue to arrive at a result that is also expressed as an RGB value. Similarly, other image representations systems, such as CYMK and others, are averaged to create a resultant value whose displayed characteristics are substantially between that of the pixel values being averaged. The formula of Eq. (1) illustrates the conventional generation of a value for pixel “x” by interpolation along a “least harmful” edge as selected from three prospective edge directions.
                    x        =                  {                                                                                          a                    +                    f                                    2                                                                              if                  ⁢                                                                          ⁢                                      (                                                                  (                                                                                                                                        a                              -                              f                                                                                                            <                                                                                                                c                              -                              d                                                                                                                                  )                                            ⋀                                              (                                                                                                                                        a                              -                              f                                                                                                            <                                                                                                                b                              -                              e                                                                                                                                  )                                                              )                                                                                                                                            c                    +                    d                                    2                                                                              if                  ⁢                                                                          ⁢                                      (                                                                  (                                                                                                                                        c                              -                              d                                                                                                            <                                                                                                                a                              -                              f                                                                                                                                  )                                            ⋀                                              (                                                                                                                                        c                              -                              d                                                                                                            <                                                                                                                b                              -                              e                                                                                                                                  )                                                              )                                                                                                                                            b                    +                    e                                    2                                                                              otherwise                  ⁢                                                                                                                      }                                    (        1        )            The current method of performing edge-directed pixel interpolation can be executed readily with minor per-pixel computational overhead, however, the resultant images are generated with distortions that may be characterized, for example, by a signal-to-noise ratio that is often insufficient for the given application.
Therefore, a method is needed to enhance edge-directed pixel interpolation to increase image quality. The present invention satisfies those needs, as well as others, and overcomes the deficiencies of previously developed general image interpolation methods.